Multiple Motion and Occlusion Segmentation with a Multiphase Level Set Method

In this paper, we propose a new variational formulation for simultaneous multiple motion segmentation and occlusion detection in an image sequence. For the representation of segmented regions, we use the multiphase level set method proposed by Vese and Chan. This method allows an efficient representation of up to 2 regions with L level-set functions. Moreover, by construction, it enforces a domain partition with no gaps and overlaps. This is unlike previous variational approaches to multiple motion segmentation, where additional constraints were needed. The variational framework we propose can incorporate an arbitrary number of motion transformations as well as occlusion areas. In order to minimize the resulting energy, we developed a two-step algorithm. In the first step, we use a feature-based method to estimate the motions present in the image sequence. In the second step, based on the extracted motion information, we iteratively evolve all level set functions in the gradient descent direction to find the final segmentation. We have tested the above algorithm on both syntheticand natural-motion data with very promising results. We show here segmentation results for two real video sequences.